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ABSTRACT 


A solution to the problem of automatic location of objects in digital 
pictures by computer is presented. A self- scaling local edge detector which 
can be applied in parallel on a picture is described. Clustering algorithms 
and boundary following algorithms which are sequential in nature process the 
edge data to locate images of objects. 
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I. INTRODUCTION 


A substantial amount of research was done in developing techniques for 
locating objects of interest automatically in digitized pictures. Drawing the 
boundaries around objects is essential for pattern recognition, tracing of 
objects in sequence of pictures for control systems, image enhancement, data 
reduction, and various other applications. References 1, 2, and 3 comprise 
a good survey of the research and application of image processing and picture 
analysis. 

Most researchers of picture analysis assumed that (1) the image of an 
object is more or less uniform or smooth in its local properties (that is, 
illumination, color, and local texture are smoothly changing inside the image 
of an object) and (2) there is detectable discontinuity in local properties between 
images of two different objects. We will adopt these two assumptions in this 
paper and assume no textural image (see Ref. 4 for an example of texture 
image analysis which does not make these assumptions). 

The work on automatic location of objects in digitized images was split 
into two branches: (1) the edge detection and edge following vs (2) the region 

growing. The edge detection meant applying in different points over the picture 
local independent operators to detect edges and then using algorithms to trace 
the boundaries by following the local edge detected. A recent survey of litera- 
ture in this area is given in Ref. 5. The region growing approach was to use 
various clustering algorithms to grow regions of almost uniform local proper- 
ties in the image. (See Refs. 6-9 for typical applications. ) More detailed 
r eferences will be given later. 

In this paper the two approaches are combined to complement each other. 
The end result is a more powerful mechanism to do the job of picture segmenta- 
tion. We developed a new edge detector and combined it with new region 
growing techniques to locate objects and thereby resolved the confusion that 
has resulted for regular edge following when more than one isolated object on a 
uniform background is in the scene (see Ref. 10). 

The contributions of this report are the following: 

(1) A new and n optimal" (given certain assumption) edge detector is 
presented. 
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(2) A simple one-pass algorithm to do region growing is presented 
which utilizes the edge detector output. 

(3) The application of path generator algorithms and "shortest path 
algorithms" to do the boundary following so as to close open 
edge lines into boundaries around regions is discussed. 

(4) Special-purpose region growing intended to close open edges 
(cracks) is described. 

(5) A special clustering algorithm which simplifies the region 
structure resulting from application of (1) through (5) is presented. 

II. DEFINITION OF TERMS 

The input is expected to be in matrix form V(i, j) i = 1,. . . , N j = 1,. . . , 
M, where V is a vector in R n , n is a function of the sensory system, usually 
1 (gray level picture), or 3 (color or x, y, z coordinates of surface in the scan 
ning direction), or 6 (color and 3-D information). An edge unit separates two 
adjacent matrix points; that is, an edge unit is between (i,j) and (i + 1, j) or 
between (i,j) and (i,j + 1) for some i,j, (see Fig. 1). 

An edge unit is usually adjacent on both ends to other edge units. There 
are 64 combinations of edge units continuing an edge unit since each of the 
edge units e^, e^, e^, e ]/» e 2 *» e 3^ * n * m ay exist or not. 

Two points on the grid (I, J) and (K,L) are said to be in the same region 
if there is a path sequence (ij.jj),. . . , suc h that i ^ = I, j ^ = J, i n = K 

and j = L, where (i m » j m ) * s adjacent to (l m+l' ^m+1^ for m 1 > . . • n - 1 and 
there is no edge unit between the two. A region will be a maximum set of 
points satisfying that property. 

An edge -line (or an edge) between region R 1 and region R 2 is the maxi- 
mal sequence of adjacent edge units such that each edge unit in the sequence 
is between two matrix points, one belonging to Rj and the other to R 2 * It is 
possible that an edge line is inside a region (R^ = R 2 ). 

An edge line which is between two different regions is called a boundary . 
An edge line which is inside a region is called a crack . An open crack is 
a crack in which at least one end terminates without connecting to any edge 
line. A closed crack is one which terminates at both ends on another edge 
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line. For instance, cracks will appear when an object is smoothly disappear- 
ing into the background on one side and has detectable discontinuity on the 
other side {Fig. 2). 

Using the above definitions, this report presents an edge detector which 
detects edge units in parallel locally on the whole image. Then a region 
grower which results in the grouping of matrix points into regions and edge 
units into boundaries and cracks is presented. A local region grower which 
tries to break a region with a crack in it into two regions for which the crack 
is part of the common boundary is then presented. Alternatively, an open- 
crack- extending algorithm is suggested to connect the open edge unit of the 
crack to another edge line. 

III. THE LOCAL EDGE DETECTOR 

The edge operator is a detector of local discontinuity in an image. When 
applied between two adjacent points such as (i, j) and (i -t 1, j), it should 
return a value which will measure the confidence that there is an edge between 
(i,j) and (i + 1, j). Since we work with noisy input to achieve reliability, the 
operator must look at two 2 -dimensional (2-D) neighborhoods Nj and to 
obtain a reliable value. Neighborhood will include (i, j) and a few adjacent 
points; N-, includes (i + 1, j) and a few adjacent points; and H = 0. As a 
result the value returns will measure the confidence that the neighborhoods 
belong to images of different objects. 

Edge detection is actually composed of three components: (1) choosing 

the proper neighborhoods, (2) the measurements of differences between image 
structures in the two neighborhoods, and (3) locking on the exact position of 
the edge. Discussion of each of these steps follows. 

IV. MEASURING DIFFERENCES IN STRUCTURE BETWEEN 

TWO NEIGHBORHOODS 

Any techniques which measure structural differences must make some 
assumption (explicitly or implicitly) on the structure of an edge vs the area 
inside the image of a region. Binford and Hershkovitz (referred to in Ref. 5) 
suggested three possible ideal edges defined by the reading profile on a 
normal -to -the -edge line (Fig. 3). 
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All of these idealized edges are in reality washed with white noise on 
both sides, where the noise is the result of both hardware noise and surface 
irregularities. Basically, the decision needed to be made is between two 
hypotheses: 

Hq: The readings in and are taken from the same object. 

H^: The readings in are taken from one object and in from 

another object. 

Neighborhoods and are the neighborhoods mentioned in the previous 
section, and the decision as to how to choose them will be described in the 
next section. 

An optimal (best for its size) decision between and Hj will utilize 
the maximum likelihood ratio as follows: Let P^ be the maximum likelihood 
estimate of the structure (reading in Nj and N^) , given that is true, and 
let be the maximum likelihood estimate of the structure, assuming is 
true. Then, 

p, 

Choose H. when = 5 — > K 

0 

?! 

Choose HL when — < K 


With probability 7, choose and with probability 1-7, choose 

p i 

H, when 75 — - K 0 < 7 < 1 

0 

This decision will be optimal for its size (see the Neyman-Pearson lemma 
in Ref. 11, page 201); hence, if the structure assumptions are valid, we have 
an ideal edge detection, given only readings in and N^. (We will deal with 
gaussian probabilities; hence we will ignore P^/P^ = K.) The conclusion is that 
Pj/Pq is the best measure of the edge strength. Following are two examples 
of applying these principles to edges of types (a) and (b) in Fig. 3. 
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EXAMPLE 1 


Assume that the edges and surfaces will be of type A, with added white 
noise which is object-dependent. Then and Hj will become 

The readings in both and are taken from the same normal 
distribution N(pq, <Tq) with unknown (jLq, cr^, 

H^: The readings on Nj are taken from normal distribution N((ij, cr^); 

and the readings on are taken from normal distribution 
N(p 2> cr^); cr j ) need not be equal to (p^, cr^). 

To apply the maximum likelihood ratio principle we need to find a 
maximum likelihood estimate for (p 0 , o-q) (p^, cr^ ) and (p,,, < t ^). Given (Xj,* * * ,x n ) 
readings taken from a normal distribution with unknown (p, cr), the 
maximum likelihood estimates are 


n 



(x.-p ) 2 

2 i=l 

cr = — 

n 



P = P, , {x. . • * * x ) 
max (p, <r) ' 1* ’ n’ 



n 



i= 1 
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2 

no* 



Hence, if the readings on are (x^,‘ ‘ ' ,x ) an d on (y^,* * * »y n ), then, 
on Np 

m 

i=i 

^1 


m 
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and on Nj combined with N^, 

m|j.j + nfa^ 

^0 = m + n 

2 2 2 2 
^ ^ ( H' q “ M* ^ ) f" ^ (M'q ~ H* ^ ^ 

*0 ™ mTn 

- m+n 

1 2 1 

P A = V*e . I 

0 ni+n m+n 



(It is convenient to work with cr since it saves computation of square roots*) 

2 

An "almost always M good threshold in this example was K = 25, That 

2 2 

is, if Pj /Pq >25, decide that there is an edge; otherwise there is no edge. 

Note that the threshold is self -scaling. In noisy or highly textured areas it 

will in effect require a higher step for an edge, and in smooth areas it will 

2 

require a lower step. In practice, we also always forced our cr to be greater 
than 0,25 since all our readings were digitized, which meant a ±0,5 random 
error in the readings. 

At this point it may be worthwhile to compare our approach with that of 
Ref, 12, Both try to use a maximum likelihood ratio to compute scores for an 
edge. But while we have a simple model and a practical way of computing the 
confidence. Ref, 12 assumes a priori deterministic classification of all possible 
idealized noise-free structures to edges and no edges. Then, for a given read- 
ing structure, the noise assumption is used to compute the probability of all 
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idealized structures that could have caused the readings. These probabilities 
are used to decide whether the readings represent on edge or not. 


EXAMPLE 2 


Here we assume that each matrix point V(i,j) is a 3-dimensional vector 

(x,y, z). Actually the raw readings are just distance R(i, j), but to avoid a 

strong dependency on the sensory position, (i,j,R) are used to compute (x,y, z). 

This is the form of input read from a device which measures distances to 

surfaces (such as radar or devices which measure the time of flight of laser 

beams to an object). The i, j corresponds to vertical and horizontal steps in 

the scanning angle. The two adjacent neighborhoods on the matrix and 

N 2 have readings (x^y^ z x ), *'* , (x n> y n , zj in Nj and (Xj ,y x , ),**', 

(x *,v , z *) in N_. We assume that objects are almost planar locally with 

' m m m 2 

added white noise. That is, if we read (Xj.y-j, Zj), * * * , (x , y , z r ) in a small 
neighborhood on an object we have a,b, c,d, o - such that 


2 

a 


+ b 


+ c 


1 


and 

ax. + by. + cz. + d + N(0, cr) = 0 i = 1, ,N 
i i l 

With this assumption the edge detection decision will be a choice between 
H Q and H^. 

Hq! The readings in the two neighborhoods are taken from the same 
plane. That is the readings on both N 1 and N 2 satisfy for some 

< a 0' b 0’ c 0’ d 0’ (T 0 ) 


where 


a Q x + b Q y + c q z + d Q + N (0, o- Q ) = 0 


a 


0 


2 




1 


for all (x,y,z) readings in and N 2 . 

There are two not necessarily equal planar fits for the readings 
on Nj and on N 2 . That is, there are (a^, b^, Cj, dj , o-j) for Nj^ 
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and (a 2 , b.,, c 2 , d^, t r^) for N 2 such that 


2,2 2 . 
a^ + bj + Cj - 1 


2 , . 2 , 2 _ , 
a 2 + b 2 + C 2 - 1 


i = l,’’’, n a. x. + b, y. + c . z. + d, + N(0, cr ) = 0 
’ li li li i i 

i = 1 , * * * , m a 2 x i^ * * c 2 Z i^ * ^2 ^(0, (r 2^ = ^ 


To apply the Neyman-Pearson principle for this case we want to 
find maximum likelihood estimates. Maximum likelihood estimates aj, b^, 
c i , d^ will be 


n 

V 1 =Z <a l X i + b l y i + C l Z i + d l ) ‘ 

i=l 


mm 

a, b, c, d 

a 2 +b 2 + c 2 =l iTl 


n 

'y' (ax, + by. + cz. + d) ; 


and 


2 v 
*1 =V 1 


Solving for the optimal (a ^ , bj, Cj, dj) is a relatively straightforward 
process. Once they are found, the maximum likelihood estimate for will be 


P, = 


1 


n_ 

2 


. e 


n n 

■\[z^ • ^1 


Hence, we have the expression which tests for an edge. It is of the following 
form: If 


m+n 


0 


n 


m 


> K 
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decide for H^, otherwise 

Note that (x,y,z) may be replaced by (i f j,g) in regular black and white pictures, 
in which case we will have a regular picture edge operation which will be able 
to handle edges of type B in Fig. 3. 

V. NEIGHBORHOOD SELECTION 


In the previous discussion on decision criteria we deliberately left out 
the question of how to choose the test neighborhoods. This is another variant 
of the properties that we want the edges to have. The edge value for a vertical 
edge between two horizontally adjacent points is taken to be the strongest case 
for an edge computed on the four pairs of neighborhoods (a) through (d) in Fig. 4. 
Taking the maximum of the maximum likelihood ratio estimate for an edge 
among the four values computed for the four neighborhoods is similar to the 
approach advocated in Ref. 13. 

A completely symmetric configuration is used to measure the confidence 
value of a horizontal edge unit between two vertically adjacent points. The 
choice of neighbors is of an experimental nature, and it worked for our problems. 
Other problem-dependent neighborhood choices are possible, and they will 
work for the specific edge structure in mind (see examples in Fig. 5). In 
choosing the size of a neighborhood, a reasonable balance between noise and 
size of object should be achieved. The bigger the neighborhoods the less 
sensitive to noise the decision will be, but the small objects may be lost. 

At this point it is worthwhile referring to the edge detector developed 
by Hueckel (Ref, 14). He found an elegant technique to compute parameters 
for step function: 


STEP 


a, b, c , d, e 


(ii j) 


d ai + bj > c 
e ai + bj < c 


For a disk 


(i.j) « DISK(i 0 ,j 0 , Y) 4 (i,j) 


(i - i Q ) + (j - j 


,) 2 < V 
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which minimizes for a given signal g(i, j) in the disk the quantity 


(i.j) 


2 


DISK 


(g(i,j)-STEP aibiCidi 



over all possible step functions. He took the parameters of a,b,c,d,e to be 
the parameters of the "best possible" edge passing through the disk. This 
measure of edge quality is clearly different from ours. Since our measure of 
edge strength is more complicated, it is unlikely that an elegant and simple 
way of finding optimal edge through a disk using our measure of edge strength 
is achievable. However, given a suggested edge structure, our approach can 
be used immediately to provide a model-driven confidence evaluation in the 
existence of the suggested edge. For the suggested (a,b, c,d, e) edge parameter 
let 


N 2 - 2 1 
* 1 * \ 
ai + bj ^ c 

(i,j) e DISK 


- d 


, 2 2 . -d) 2 /N 2 

ai + bj > c 
(i,j) € DISK 


N,= 2 l 

ai + bj < c 
(i.j) e DISK 


7^ (g (i.j) - e) 2 /Nj 

ai + bj < c 
(i,j) e DISK 
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Then, 


M-o = < N 2' ^2 + N l- ^i)/ N 1 +N 2> 

2 = T. (g d.j) - i*n> 2 / N o 

o zzz 

(i> j) e DISK 



VI. LOCKING ON A DETECTED EDGE 

Computing the edge value is not usually sufficient to decide where to put 
the edges. The values that are computed usually look like the ones in Fig. 6, 

One way of forcing the edge to be well defined is to constrain it to be a 
local maximum in addition to having a confidence value higher than a certain 
threshold. This is, of course, extremely important for locking on the center 
of the edge. Usually there is still some local ambiguity on the location of the 
edge, and for many practical reasons it is better to treat the area around an 
edge as ambiguous. The source of problems here is that, because of computing 
time constraints, it was impossible to find a global optimum for edge lines 
using all available data, and it was necessary to use only local information 
for evaluating edge units in this level. In our system, the decision as to where 
exactly to put the edge was left for the region grower (see below). To demon- 
strate the possible 2-D ambiguity, see Fig. 7. 

The search for a maximum may be used for special-purpose edge detec- 
tion. For instance, if we look only for one dark stripe crossing a white back- 
ground, forcing the edge to be the absolute maximum or minimum on a hori- 
zontal line in the image (keeping track of signs of change) will supply the 
appropriate pair of edges. 
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VII. REGION GROWING 


The output of an application of an edge detector results in two new 
matrices in addition to the matrix V (i,j) of raw data. The first is EV(i, j), 
which is the measure of the confidence that there is an edge unit between (i, j) 
and (i, j + 1); the second is EH(i,j), which measures the confidence that there 
is an edge unit between (i,j) and (i + l,j). EV(i,j) and EH(i,j) may include 
extra bits as determined by the direction of the change on that suggested edge 
unit. 

This output as it stands is not sufficient for application of pattern recog- 
nition and various picture quantitative analysis tasks, since outlines of objects 
are needed in order to recognize features. Hence a region grower which will 
outline objects is needed. A straightforward approach is to classify as edge 
units all edge units where EV(i,j) or EH(i, j) are of confidence value greater 
than some threshold T and which are local maxima. A point (i^, j^) is a local 
maximum if EV(i 0 , j Q ) > E V(i Q ,j 0 + 1) and EV(i Q , j Q ) >EV(i Q , j Q - 1) or EH(ij, j Q ) > 
EH(ip - 1 , j q) and EHfi^jg) > EH(ig + 1 , j q ). Unfortunately, this straightforward 
approach fails. Indeed, it creates an excellent display of boundaries for the 
viewer, but as a result of image irregularities too many cracks are generated 
and too many skinny regions appear on boundaries of objects. Because of that, 
a sequence of algorithms is called to utilize more global structure to find 
exact positions of boundary lines and eliminate most cracks and regions which 
are too small to be of interest. 

We start by describing a one-pass algorithm which parses the edge data 
into data structures of regions, boundaries, closed cracks, and open cracks 
and creates, as byproducts, two arrays, FH(i, j) and FV(i, j), where FH(i, j) 
means that the program puts an edge unit between (i - 1 , j ) and (i, j), and 
FV(i, j) means an edge unit between (i, j) and (i,j - I). 

i 

To ease the description of the decision mechanism for putting edges, we 
need to define a few new terms. Let T > 0 be the edge confidence threshold; 
then, 

(1) 'd 1 is the distance between two adjacent grid points. It will be 

d((i, j), (i - l,j)) A d((i - l,j), (i,j)) A if EH(i,j) < T 
then 0, else EH(i,j) 

d((i,j), (i,j - 1)) 4 d((i, j - 1), (if j)) 4 if EV(i,j) < T 
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then 0, else EV(i,j) 

(2) Reg(i,j) will be the region to which the point (i, j) belong. 

(Reg(i.j) is not defined to all points until the program is finished.) 

(3) Val (i,j) = M in {d(( i,j) t (k .l))} 

l i-k | + |j-l | = 1 

No edge unit between 
(i.j) and (k. 1) 

This value will be + « if (i.j) is the only point in its region. 

(4) VaKRegj) = Min(Val (i.j)) 



Reg(i.j) = Reg 1 

(5) A point P will be the minimum point for its region if 

Val(P) = Val(Reg(P) ) 

The algorithm is designed so that at each state there is always a non- 
decreasing path from each minimum of any region to any other point in the 
region and the path enters that point from its minimum direction. 

That is, if P and Q are two points such that 

Reg(P) = Reg(Q) and Val(P) = Val(Reg(P)), then 

there is a path (x^, x^,* ' , x^) such that 

(a) Xj = P, = Q 

(b) Reg(x.) = Reg{P), i = 1,* * * ,n 

(c) x. adjacent to x. + 1> d(x. +2 , x. + 1 ) >d(x. + 1> x £ ) 

(d) d(x , x ,) ~ Val(x ) 

n n-1 n 

We say if such a path exists that Q is reachable from P. 

That is, two points are in the same region if you can get from one to the 
other in a path which does not cross a ridge of edge values, 

VIII. ALGORITHM DESCRIPTION 

The program scans the image from left to right, line by line. That is, 
the scanning is such that when point (i , j ) is processed, the program already 
worked on all points (i^, j^) such that (jj<j) or (j = and i^ <i). 
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Assume the program is processing point (i, j). 

Let Dj be a Boolean variable set to true if this program is not going to 
put an edge unit between (i,j) and (i, j-1), and false otherwise, and let be a 
Boolean variable set to true if the program is not going to put an edge unit 
between (i, j) and (i - 1, j), and false otherwise. Let Rj = Reg(i,j-1) and 
R 2 = Reg(i-1 , j). (see Fig. 8). 

The decision on the values of Dj and D 2 is described by the following 
ALGOL-like program: 

Begin 

Boolean Good- Down^ , Bad-Down^, Up^, Good-Down 2> Bad-Down^, Up 2 ; 
Good-Down^ — if d((i, j), (i, j-1) < Val(i, j-1) A Val(i, j-1) < Val(Rj) 
then TRUE 
else FALSE; 

Comment : Good-Down^ is true if point (i, j) is going to become a new 
minimum for R^ (the region above). And it is adjacent to an old minimum; 
hence, any point of R^ reachable from the old adjacent minimum will be 
reachable from the new; 

Bad-Downj — if d((i,j), (i, j - 1 )) < Val(i,j-1) A (Val{i,j-1) >Val(R 1 )) 
then TRUE 
else FALSE; 

Comment : This variable is true if (i, j) is not reachable from all minima 

of Rj going through (i,j-l); 

U Pl - if d((i,j), (i, j-1)) > Val(i, j-1) 
then TRUE 
else FALSE; 

Comment : This variable is true if point (i, j) is reachable from any 

minima of by continuing the path that leads from that minimum to 

(i. j-i); 

Good-Down 2 - if d((i, j), (i- 1 , j)) < Val(i-1 , j) A (Val(i-1, j) < Val(R 2 )) 
then TRUE 
else FALSE; 

Comment : This variable is true if point (i, j) is going to be a new minimum 

for R 2 (the region minimum, to the side) and is adjacent to an old 
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minimum of R 2 ; hence any point reachable from the adjacent old mini- 
mum will be reachable from (i, j); 

Bad-Down^ - if d((i,j), (i- 1 , j)) < Val(i- 1 , j) A Val(i- 1 , j) > Val(R 2 ) 
then TRUE 
else FALSE; 


Comment : This variable is true if (i, j) is not reachable from all minima 
of R^ through (i-l,j); 

Up, — if d(i,j), (i-1 , j) > Val(i-1, j) 
then TRUE 
else FALSE; 


Comment : This variable is true if point (i, j) is reachable from any minima 

of R^ by continuing the path that leads from that minimum to (i-1, j); 

If Good-Down^ A Good-Down^ then •*- true 

else 

If Good-Down^ A Bad-Down^ then begin D { *- true; false; end 

else if Good-Down^ A Up 2 then begin 


if d((i, j), (i, j - 1 )) > d(i(i,j), | 

(i-1, j)) 

then true 


else begin •- true; 

D 

end 



else if Bad-Down^ then begin if Good-Down 2 

Dj — false 
true 

end 

else begin 
4- false 

false; end 

end 

else if Upj A Good-Down^ then begin 

if d«i,j), (i-1, j)) >d«i,j), (i, j - 1 ) ) 
then Dj D 2 true 
else begin <- true; 

end 


- false; end; 

V Up 2 then begin 


false; end 


else if Upj A Up 2 then, if R^ = R^ then D 2 true else 

if d((i,j), (i-1, j)) > d((i, j), (i,j-l)) 
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then begin truej D^, false; end 

else begin «- false; ♦- true; end 

end 

Comment: Only one of D and can be true; otherwise we cannot 

1 ^ 

guarantee entrance through minimum value from all minima of both Rj 
and 

else if Upj A Bad-Down^ then begin *- true; false; end 

Val(i,j) «- «•; 
if then begin 

Val<i,j-1) - Min(d((i,j), (i,j-l)), Val(i,j-1)); 

Val(i,j) - d((i,j), (i, j-1)); 

Val(Rl) - Min(Val(R 1 ), Val(i,j)); 
end; 


if D then begin 

Val(i-lJ) - Min(d((i,j), (i-1, j)), Val(i-l,j)); 
Val(i, j) - Min(Val{i, j ), d((i,j), {i-l,j))); 
Val(R 2 ) - Min(Val(R 2 )), Val(i.j)); 
end; 

If not (Dj V D 2 ) then Val{Reg(i, j)) oo • 


The and e., (see Fig. 8) may exist or not, and as a result there are 
four starting conditions. The program may put Dj, D 2 , and D 2 or none 
of them, and hence, there are 16 cases in a point. (See Fig. 9 for a brief 
description of the different cases.) 

Merging of two regions may always result in transformation into a crack 
of a previously common boundary of the two regions. In general, each operation 
of the region grower is fairly elaborate: more than meets the eye. The data 
structure used is not described in this paper, but it is essentially the same 
data structure described in Ref. 9, with slight modification to include edge 
line representation through chain encoding. 

This one-pass algorithm is local and requires relatively small core 
resident data. However, it does not create maximal regions with respect to 
our criteria of path connectivity and reachability. The reason is the possible 
asymmetry of the distance function. On the other hand, it is relatively simple 
and fast when other algorithms are considered. The maximality problem may 
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be easily corrected if backup is allowed. Note also that, in fact, the threshold 
T plays a very small role in defining the output of the algorithm. 

IX. SIMPLIFICATION OF THE RESULT OF BASIC REGION GROWING 

There are two straightforward options for simplifying the output of the 
one-pass region grower: (1) take all regions that are too small to be interesting 

and melt them into their closest neighbor (the distance between two regions 
will be defined later in the paper); (2) take all short cracks which are weak 
(strength of the edge line will be defined later) and delete them. Of course, the 
threshold below which a crack is weak and a region is small is a function of 
how much we want to elaborate the task of the image analysis and is defined 
heuristically. In fact, in the current implementation all cracks are deleted 
since the edge operator was sensitive enough for our purposes. 

X. GROWING OPEN CRACKS INTO CLOSED CRACKS 

One possible way of closing open cracks is to grow them in length from 
their open end until the extended edge line meets already existing edge lines 
and closes. On the open end in each step there are three choices as to where 
to extend the edge line: go straight ahead, turn left, or turn right. The 
decision as to which direction to take will be to minimize the cost of closing 
the open crack, where the cost is defined heuristically. One possible choice 
is as follows: 

Given the original crack, define two distributions which will describe 
the properties on either side of the crack, P^ and P^. The cost for adding 
an edge unit will be the maximum likelihood ratio between the two assumptions. 

H^: the two sides of the edge unit belong to the same side of the crack 

(the best choice between on both sides of the extension and 
on both sides). 

: there is a different distribution on either side chosen according to 

geometrical constraint (Fig. 10). 

Cost = P^j /Ptt 
H 0 H 1 
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Note that since the cost function is additive it can be used in conjunction 
with the shortest path algorithm (Ref. 15, Ch. 3) to find the nearest (least 
expensive) path to a closing edge unit. Reference 16 describes another heuristic 
of line expanding which may be applicable to our case. 

XI. BREAKING A REGION INTO TWO AROUND A CRACK 

An alternative approach to breaking a region into two regions so as to make 
the crack into a part of a boundary is to use special-purpose region growing. 

Assume that there is a crack in a regular gray level picture (that V(i, j) e R. ), 

2 1 

with readings with mean (a.j and variance on a small neighborhood on one 

side of the crack and mean and variance <r^ on the other side. Assume 

that the crack is inside region R; then we can break the points in R into two 

classes, C^ and C~,: 


C 


1 


A 


(i,j) 


1 




e 


1 

2 


/v(i,j) - p,Y^ 

Hr 8 -) 


C 2- R ~ C 1 


Then in some sense we would expect to be on the first side of the 
crack and on the second side of the crack. Unfortunately, it may turn out 

that Cj or are not path-wise connected. As a result, one of the connected 

components which border on the crack should be picked out. A more heuristic 
approach is to grow a region around each of the two sides of the crack, and 
to stop when a new point has a neighborhood which is more likely to belong to 
the other side. Then take the smaller of the two regions resulting and make 
it Cjj then will be 


C 


2 " 


R - 


C 


1 
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This algorithm can be used also to allow flexible human interaction in 
analyzing the scene. 


XII. MERGING REGIONS 

The basic region grower utilized local detection procedures. Better 
decisions are achievable (at least theoretically) by using more global informa- 
tion. The problem is how to allow this additional information and still keep the 
program lean and fast. Research in that area was reported (Ref. 16). Basically, 
our approach is to be oversensitive on the local pass and as a result to over- 
segment the picture. But then we take the data output (which is reduced data) 
and simplify it. We take pairs of regions with common boundaries and merge 
them into one. In order to do that reliably, a confidence value which measures 
the confidence that the pair of regions are different is computed, and iteratively 
we pick the pair of regions with the lowest confidence of being different in the 
current structure, merge them, and update the structure. The confidence is 
computed as the product of two components: (1) edge line strength (on the 

common boundary of the two regions) and (2) the difference of the properties 
inside the region. Both of these values are computed on the basis of assumptions 
similar to those used in the edge confidence evaluation. For instance, if we 
assume gray level readings, at each point (i, j) the value is a positive integer 
value. Along the edge line take two small neighborhoods on the two sides 
(like a 4-point-wide stripe on each side of the edge) and assume that the readings 
in one neighborhood are 


(x t ) 


n 

i=l 


and 


in the other. Then the edge strength will be the ratio between the maximum 

likelihood estimate that (x.) n (x! ) m , are from two different normal distribu- 

i i= 1 11= 1 

tions to the maximum likelihood estimate that (x.). , (x. ). , are taken from the 

l i=l l i=l 

same normal distribution. The computation technique for the values is the 
same as that used in the edge evaluation model 1. This value gives the boundary 
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strength evaluation; a similar value is computed on the basis of distribution in 
each region, which gives the difference in properties of the two regions. 
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Fig. 1. Edge unit structure 
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READINGS 


(c) 


— EDGE AREA 


DISTANCE, NORMAL TO EDGE 


(a) IDEALIZED STEP EDGE (DOMINANT EDGE TYPE IN VISUAL IMAGES). 

(b) PURE GRADIENT EDGE (CORNERS ARE ESPECIALLY FREQUENT IN ANALYSIS 
OF 3-D IMAGES WHEN DIRECT MEASURE OF DISTANCE IS AVAILABLE) 

(c) SPIKE EDGE (APPEARS FREQUENTLY IN CORNER EDGES IN VISUAL IMAGES). 

Fig. 3. Typical edges 
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Fig. 4. Typical neighborhoods for edge detection 
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GENERAL EDGE ORIENTATION 
DETECTOR IN DISK COM- 
PUTED FOR FINDING OPTIMAL 
ORIENTATIONS, DEPENDING 
ON THE DESIRED ANGULAR 
RESOLUTION 


CORNER DETECTOR; T CORNER DETECTOR 

a' AND <t TAKEN 
TO BE TYPICAL 
ANGULAR RESOLUTIONS 


Fig, 5, Extended neighborhoods set 
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Fig, 6, An ideal edge value cross section 
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THE (i, j) ARE POINT NUMBERS AND THE VALUES ARE EDGE UNIT 
VALUES. CLEARLY POINTS (I, 1) (1, 2) (1, 3) (2, 1) (3, 1) SHOULD 
BE IN ONE REGION AND (3, 2) (3, 3) (2, 3) IN ANOTHER , BUT 
WHERE (2, 2) SHOULD BE IS TOTALLY AMBIGUOUS (ASSUMING 
THAT SINGLE POINT REGIONS ARE NOT ALLOWED). 

Fig. 7. Region growing ambiguity example 


• 0, i - 0 


(i - 1, i) 


('/ i) 


Fig. 8. Algorithm terms definition 
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Fig. 9. The different region growing decisions 
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THE 3 OPTIONS TO EXTEND AN OPEN CRACK AND THE 
CORRESPONDING ASSUMPTION ON DISTRIBUTING 

Fig. 10. The three options to extend an open crack and the 
corresponding assumption on distributing 
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